A light field represents the amount of light passing through each point of a three-dimensional (3D) space along each possible direction. It is modelled by a function of seven variables representing radiance as a function of time, wavelength, position and direction. In Computer Graphics, the support of light field is reduced to four-dimensional (4D) oriented line space.
The acquisition of 4D light-field data, which can be viewed as a sampling of a 4D light field, i.e. the recording of light rays, is explained in the article “Understanding camera trade-offs through a Bayesian analysis of light field projections” by Anat Levin and al., published in the conference proceedings of ECCV 2008 is an hectic research subject.
Compared to classical two-dimensional or 2D images obtained from a conventional camera, 4D light-field data enable a user to have access to more post-processing features that enhance the rendering of images and the interactivity with the user. For example, with 4D light-field data, it is possible to perform refocusing of images with freely selected distances of focalization meaning that the position of a focal plane can be specified/selected a posteriori, as well as changing slightly the point of view in the scene of an image. In order to acquire 4D light-field data, several techniques can be used. For example, a plenoptic camera is able to acquire 4D light-field data. Details of the architecture of a plenoptic camera are provided in FIG. 1A. FIG. 1A is a diagram schematically representing a plenoptic camera 100. The plenoptic camera 100 comprises a main lens 101, a microlens array 102 comprising a plurality of micro-lenses 103 arranged in a two-dimensional array and an image sensor 104.
Another way to acquire 4D light-field data is to use a camera array as depicted in FIG. 1B. FIG. 1B represents a multi-array camera 110. The multi-array camera 110 comprises a lens array 112 and an image sensor 114.
In the example of the plenoptic camera 100 as shown in FIG. 1A, the main lens 101 receives light from an object (not shown on the figure) in an object field of the main lens 101 and passes the light through an image field of the main lens 101.
At last, another way of acquiring a 4D light field is to use a conventional camera that is configured to capture a sequence of 2D images of a same scene at different focal planes. For example, the technique described in the document “Light ray field capture using focal plane sweeping and its optical reconstruction using 3D displays” by J.-H. Park et al., published in OPTICS EXPRESS, Vol. 22, No. 21, in October 2014, may be used to achieve the acquisition of 4D light field data by means of a conventional camera.
There are several ways to represent 4D light-field data. Indeed, in the Chapter 3.3 of the Ph.D dissertation thesis entitled “Digital Light Field Photography” by Ren Ng, published in July 2006, three different ways to represent 4D light-field data are described. Firstly, 4D light-field data can be represented, when recorded by a plenoptic camera by a collection of micro-lens images. 4D light-field data in this representation are named raw images or raw 4D light-field data. Secondly, 4D light-field data can be represented, either when recorded by a plenoptic camera or by a camera array, by a set of sub-aperture images. A sub-aperture image corresponds to a captured image of a scene from a point of view, the point of view being slightly different between two sub-aperture images. These sub-aperture images give information about the parallax and depth of the imaged scene. Thirdly, 4D light-field data can be represented by a set of epipolar images see for example the article entitled: “Generating EPI Representation of a 4 D Light Fields with a Single Lens Focused Plenoptic Camera”, by S. Wanner and al., published in the conference proceedings of ISVC 2011.
Light-field data can take up large amounts of storage space, up to several Tera Bytes (TB), which can make storage cumbersome and processing less efficient. In addition light-field acquisition devices are extremely heterogeneous. Light-field cameras are of different types, for example plenoptic or camera arrays. Within each type there are many differences such as different optical arrangements, or micro-lenses of different focal length sand, above all, each camera has its own proprietary file format. At present there is no standard supporting the acquisition and transmission of multi-dimensional information for an exhaustive over-view of the different parameters upon which a light-field depends. As such, acquired light-field data for different cameras have a diversity of formats. The present invention has been devised with the foregoing in mind.